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In recent years, artificial intelligence applications for understanding the human thinking process and 
transferring it to virtual environments come into prominence. The fuzzy logic which paves the way for 
modeling human behaviors and expressing even vague concepts mathematically, and is also regarded 
as an artificial intelligence technique has become one of the most preferred methods in the solution of 
decision problems. Numerous decision-making situations are faced in education as well. Particularly, it 
is rather challenging and requires experience to decide in a fair way and assessing students' 
performances without making any error in the process of assessment and evaluation. The purpose of 
the current study is to assess students' performance with the fuzzy analytic hierarchy process (FAHP), 
one of the multi-criteria decision making methods based on the fuzzy logic approach. The formation of 
the proposed system on the basis of fuzzy set theory determines that it can provide benefits in 
modeling these ambiguities in human mental processes and also it can reach fairer, more sensitive and 
objective results. Being used especially in making important decisions in companies and in developing 
smart vehicles in engineering, FAHP methods have brought into question whether this method can be 
used in education or not. This study reveals that FAHP method can be used in the evaluation of 
students' projects in education. 

Key words: Performance evaluation, fuzzy multi criteria decision making methods. 


INTRODUCTION 

Decision-making is a phenomenon encountered 
constantly in every areas of human activities. Decision¬ 
making process is about selecting the most suitable 
alternatives according to certain criteria in occasions that 
one faces with existing alternatives. This process is 
considered to be a tough one for decision makers 
because of its uncertainty and subjectivity (Bai and Chen, 
2008; Lin, 2010). Today, studies on this phenomenon 
appearing in many fields like management, industry, and 


education have gained a different dimension with the 
advancing technologies. Especially in recent years, 
artificial intelligence applications on human thinking 
process and adapting this system into computer 
environment had gained importance. Allowing human 
behaviors to be modeled and vague concepts to be 
expressed mathematically, fuzzy logic, also is considered 
to be an artificial intelligence technique, have become a 
frequently preferred method in solving problems in 
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decision-making process. Education is one of the 
application areas of fuzzy logic which has a wide range of 
application areas (Ibrahim, 2001; Kavcic et al., 2003; 
Kinshuk et al., 2001). There are many occasions requiring 
numerous decision-making process in education. 
Particularly, making right decisions in assessment and 
evaluation process and evaluating students correctly are 
hard processes requiring experience (Bai and Chen, 
2008). 

Project evaluation process includes the evaluation 
activities regarding the students’ levels of carrying out 
knowledge or skill (Nitko, 2001). Generally, numerical 
values of verbal expressions are used in the evaluation of 
students’ projects. However, it is observed that certain 
numbers are used by the decision-makers because of the 
difficulties that they face in the process of digitizing the 
verbal (qualitative/linguistic) grades (Kahraman et al., 
2007). The literature indicates that instead of the use of 
certain numbers in expressing human feelings and 
decisions in the decision-making process, the use of 
linguistic variables which is a more realistic option, allows 
one to carry out a better, right and comfortable evaluation 
(Chan et al., 1999; Gu and Zhu, 2006; Kahraman et al., 
2007; Lin, 2010). 

The process of students’ project evaluation includes 
some uncertainty and subjectivity like in other decision¬ 
making processes (Bai and Chen, 2008; Law, 1996; Wu, 
2003). Also, another occasion affecting the decision¬ 
making in this process is the outnumbering evaluation 
criteria. Various scientific methods have been developed 
with the aim of finding a solution to the cases having high 
number of criteria in decision-making problems. One of 
the solution methods is multi-criteria decision-making 
model (Duran and Aguilo, 2008). Multi-criteria decision¬ 
making model is concerned with structuring and choosing 
an option from a set of alternatives which are 
characterized by different criteria. The decision maker 
ranks for these criteria by proposing the importance. 
There are various multi-criteria decision-making 
approaches in the literature. Most popular among them 
ELECTRE, Analytic Hierarchy Process (AHP), Technique 
for Order Preference by Simulation of Ideal Solution 
(TOPSIS), Preference Ranking Organization Method for 
Enrichment Evaluations (PROMETHEE), and Analytic 
Network Process (ANP). These techniques and 
approaches have been suggested to choosing the best 
alternative. The benefits of AHP compared with other 
multi-criteria decision making methods is its ability to its 
simplicity, flexibility, accuracy, ease of understanding 
(Forman and Gass, 2001) and include intangibles (Harker, 
1987). Since multi-criteria decision-making model 
depends more on qualitative data and human thoughts, 
nowadays, fuzzy logic, which is more suitable to evaluate 
such data, is frequently used (Bozbura et al., 2007; 
Mardani et al., 2015). Fuzzy logic is an effective way of 
explaining the uncertainty in the decision-making process 


(Lin et al., 2007) and its qualities (Pedrycz and Gomide, 
1998) by enabling an evaluation with the verbal variables. 
In these methods called fuzzy multi-criteria decision¬ 
making models, fuzzy numbers including uncertainty and 
reflecting human thoughts which cannot be distinguished 
from each other are used. The literature on fuzzy multi¬ 
criteria decision-making method suggests that there have 
been many studies in different fields (Mardani et al., 
2015). Each multi-criteria decision-making method has its 
own privilege, strength, and weakness for certain 
applications (Zare et al., 2016). It is an emphasized topic 
by different researchers that with the integration of fuzzy 
set theory and Analytic Hierarchy Process (AHP) which is 
one of the multi-criteria decision making methods, highly 
sensitive and right decisions can be achieved (Fu et al., 
2006; Ong et al., 2003; Yang and Chen, 2004). Fuzzy 
AHP is more representative for decisions of humans 
(Cheng et al., 2008). The fuzzy AHP method has been 
widely used by various researchers to solve different 
decision-making problems. Nagpal et al. (2015) used 
fuzzy AHP to compare and rank different websites of an 
educational institute on their usability criteria. Chen et al. 
(2015) present a framework for teaching performance 
evaluation based on fuzzy AHP and fuzzy comprehensive 
evaluation method. Jie (2010) applied fuzzy AHP to 
evaluate online course quality. Chao and Chen (2009) 
used fuzzy AHP to examine E-learning system 
effectiveness accordingly to send the results back to 
managers in schools. Lin (2010) adopted an evolution 
model for evaluate course website quality. However, it 
has rarely been applied in the field of evaluation process 
in education. Therefore, the aim of this study is to 
evaluate the student projects and interpret the results of 
the evaluation with Fuzzy AHP, one of the multi-criteria 
decision making process. 

Fuzzy set theory 

In daily life, it is impossible to make a certain definition of 
many occasions. The reason for this is the high degree of 
uncertainty in real life. In order to define effectively 
subjective judgment or ambiguous problems via linguistic 
variables, fuzzy set theory was proposed by Zadeh 
(1965) on the uncertainty of human thought, for the first 
time. An object is either an element of the set or not in 
classical set theory. In no circumstances, partial 
membership can be discussed. If the membership value 
is 1, it is the full element of the set; if it is 0, it is not the 
element of the set. In contrast to classical sets, the 
membership degrees of the elements can vary in infinite 
numbers between the range of [0,1] in fuzzy sets. 


Membership function 

Fuzzy sets are defined by membership functions (Zadeh 
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and Kacprzyk, 1992). The membership function of A 
fuzzy set is shown by pA(x). Fuzzy sets described each 
object with the membership function having the degree of 
membership ranging between 0 and 1 (Zadeh, 1965). If x 
element definitely belongs to A fuzzy set, it is pA(x)=1; if it 
does not definitely belong to, it is pA(x)=0. In fuzzy sets, 
there are no precise limits; instead, there is a gradual 
transition depending on the case of belonging to the set 
or not, and this transition is described with the 
membership functions (Klir and Yuan, 1995). Although 
there are a large number of membership functions, 
generally triangular, trapezoidal, gaussian, and bell¬ 
shaped membership functions are used. In the current 
study, triangular membership function is used. Although 
there are a large number of widely membership function 
which include triangular, trapezoidal, gaussian, and the 
bell curve, triangular membership function was used in 
this study. 


Verbal /linguistic variables 

In fuzzy logic, verbal/linguistic variable is as an important 
concept of fuzzy sets. Linguistic variables are used to 
express human’s feelings and decisions (Chan et al., 
1999). Since human judgments are generally vague and 
cannot be estimated via precise numeric value, precise 
values remain insufficient in modeling the real life in 
many occasions. According to Zadeh (1965), linguistic 
variables are used to avoid excessive complexity. The 
value of linguistic variables in natural languages is not 
numbers but words or sentences; and decision-making 
with words or sentences is easier than decision-making 
with numbers. The studies in the literature indicate that 
the evaluations via linguistic variables are more 
comfortable for the decision-makers and more realistic 
results are revealed (Chu and Lin, 2003; Gu and Zhu, 
2006; Zhang and Lu, 2003). Since linguistic variables can 
be analyzed qualitatively and can be used by grading in a 
certain range instead of single value, it allows obtaining 
more sensitive results (Lin et al., 2007). 


Fuzzy numbers 

Fuzzy numbers are a fuzzy subset of real numbers. 
Fuzzy numbers are used to handle the indefinite 
numerical values such as around 7 or close to 10 (Chen 
and Hwang, 1992). There are fuzzy number types such 
as triangular, trapezoidal, and bell shaped curve. 
Generally triangular fuzzy numbers are used in studies. 
The triangular fuzzy numbers are described via tree real 
numbers (I, m, u). The membership factor is defined by 
depending on these numbers. The membership function 
of the triangular fuzzy number is shown in Figure 1. 

The real number values (I, m, u) constituting the 
triangular number are “I”, the smallest probable value, 





Figure 1 . The membership function of the triangular fuzzy 
number. 


“m”, the most probable number, and “u”, the largest 
probable value. The membership function of a triangular 
fuzzy number is defined as follows. 


0 , 

(x-l)/(m-l), 

(u-x)/(u-m), 

0 , 


x<l 

l <X< 111 

m < x < u 
x>u 


( 1 ) 


Fuzzy analytic hierarchy process 


AHP is one of the multi-criteria decision making methods 
which are widely used in modeling unstructured problems 
arisen in fields like politics, economics, social and 
management sciences (Saaty, 1980). AHP is also 
employed to solve complex decision problems involving 
subjectivity (Saaty, 1990). It is argued that AHP remain 
incapable in reflecting human thought system completely 
and dealing with ambiguity and uncertainty occurring in 
the process of pair wise comparison although calculation 
is based on information given by decision makers 
(Buyukozkan et al., 2004; Deng, 1999; Kahraman et al., 
2003; Lin, 2010). Therefore, recently, by combining AHP 
and fuzzy theories, some studies have been carried out 
in order to determine the rate of criteria inferring from 
subjective perceptions (Fu et al., 2006; Mardani et al., 
2015; Yang and Chen, 2004). This method called Fuzzy 
Analytic Hierarchy Process (FAHP) employs value 
ranges, instead of precise numbers, in determining the 
rates of pair wise comparisons (Bender and Simonovic, 
2000). The power of representing the vague situations in 
the process increases as a result (Bozbura et al., 2007; 
Lin, 2010). 


Essences of fuzzy analytic hierarchy process 

Fuzzy AHP reduces complex problems to simpler piece 
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of problems by constructing a hierarchical structure and 
hence, it allows solving of the problem in a shorter time. 
Since it takes both qualitative and quantitative factors into 
consideration and has an easy and simple way of use, 
this method which analyzes pair wise comparisons, 
options and criteria in terms of their significance and 
dominance is employed frequently in solving complex 
decision problems. Stepwise procedure of “Extended 
Analysis Method” developed by Chang (1996) is as 
follows: According to Chang’s method, each target is 
taken and each dimension analysis is respectively 
applied. In this way, for each dimension m dimension 
analysis is obtained. Here, all described M j gi variables as 
I, m and u are triangular fuzzy numbers. 


decision is chosen: 

V (M > M^M^.-.Mk) = V[(M > M,) and (M > M 2 ) and ... (M 
SM k )] 

= min V(M > M,), i=1,2,3,...,k 

Vk= 1,2,3.k 

k =£ i (6) 

Step 4: If d'(Ai) = min V (Si >S k ) , priority vector is 
presented in Equation 7 

W' = (d'fAO, d'(A 2 ), d'(A n )) T (7) 

Ai (i=1,2, ..., n) 


MV M 2 gi ,...M m gi , i= 1,2,3,...,n (2) 

Step 1: To the goal denoted as i., fuzzy synthetic 
dimension value is shown as follows: 


m 

S.-Z 


n m 

IZm 

_i= 1 7=1 


-1 


( 3 ) 


Step 5: Normalization is a mathematical calculation done 
for reducing each criterion to the range of [0,1] and 
allowing to compare the results. With normalization, 
normalized weight vector is presented in Equation 8. 
Here, W is not fuzzy but priority vector composing of real 
numbers. 


W= (d(Ai), d(A 2 ),... ,d(A n )) T 


( 8 ) 


Step 2: Priority values of decision elements in hierarchy 
are determined by comparing described synthetic values. 
However, since synthetic values are triangular fuzzy 
numbers, while making comparisons, the following rules 
should be considered: 

Let M,= (h, m,, u-i) and M 2 =(l 2 , m 2 , u 2 ) be two triangular 
numbers, the degree of possibility of equation M 2 > M, is 
shown in Equation 4. 

\/(M 2 > M,) = sup y >x[min(g M i(x), MMz(y))] (4) 

This equation is based on the assumption of constructing 
a set with weak ones of fuzzy correlation between p M1 
and Pm 2 choosing the strongest of all. This equation is 
shown in Equation 5. 


A CASE STUDY 

This study is conducted with the aim of evaluating the projects 
developed by university students within a course and choosing the 
best project. First of all, the hierarchical structure of the project 
evaluation model is constructed in line with expert opinions, and 
then the main criteria and sub criteria under each main criterion in 
this structure are identified. The degree of importance of each 
criterion is determined by the expert carrying out the evaluation. 
Lastly, the projects are evaluated via linguistic expressions in the 
lights of the set criteria. After the evaluation via linguistic variables, 
the linguistic variables are converted into fuzzy numbers and the 
data are analyzed via FAHP method. In the evaluation process with 
FAHP, Extended Analysis Method (Chang, 1996) is used and the 
paired comparison of the criteria and projects are made by the 
system. After the required processes completed on the weight of 
each criterion and the project’s score of students, the conclusion is 
reached. The project evaluation process is summarized in Figure 2. 


V (M 2 ^ M-i) = Pm 2 (d) = 


1 , if m2 > ml 

0 , if ll > u2 


l\-u2 

(m2 - u2 ) - (ml - ll) 


, otherwise 


( 5 ) 


Step 3: Equation 6 is used in order to calculate the 
degree possibility for a fuzzy number to be greater than k 
fuzzy numbers. Pairs of fuzzy numbers are compared 
and the results are obtained. Among these results, 
minimum value denoted as d'(Aj) belonging to each 


Forming the hierarchical structure 

Forming the hierarchical structures in the analysis of complex 
systems allows easier access to the desired destination (Chang, 
1996). In FAHP method, the aim is at the top of the hierarchy and 
under this aim, there are respectively main criteria, sub criteria, and 
options. The hierarchical structure in the evaluation of students’ 
projects is indicated in Figure 3. 


Determining the project evaluation criteria 

Studies show that experts play an important role in determining the 
evaluation criteria (Ma and Zhou, 2000). Professors generally have 
expectations about the course achievements and experiences so 
they identify the evaluation criteria in line with this framework. The 
project evaluation criteria were pilot tested with five experts to 
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Figure 2. The project evaluation process with FAHP. 
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Figure 3. The hierarchal structure of criterion and options. 
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validate the instrument. These experts have more than 10 years’ 
experience in instructional technology education and they have 
varying degrees of experience in project evaluation. They were 
asked to opinions on the meaningfulness, relevance, and clarity of 
the criteria. Therefore, the evaluating criteria have confirmed 
content validity. Based on five experts’ feedback the four main 
criteria content, design, technical and presentation are identified as 
most crucial for evaluating the projects. 


The weights of the criteria used for the project 

The criteria are weighted via linguistic expressions. The weights of 
the criteria in project evaluation are determined by the decision 
maker by using linguistic variables as follows “Very Important (V.l.)”, 
“Quite Important (Q.l.)”, “Important (I.)”, “Slightly Important (S.I.)”, 
and “Rarely Important (R.l.)”. The criteria used in the evaluation of 
students’ projects and the linguistic expression of the weights of 
each criteria are presented in Table 1. 


Evaluation of the students’ projects 

Rather than grading with precise numbers, the students’ projects 


are evaluated via linguistic variables used in ambiguous occasions. 
The professor of the course uses a linguistic scale such as “Very 
Good (V.G.)”, “Good (G.)”, “Average (A.)”, “Poor (P.)”, and “Very 
Poor (V.P.)”. In the project evaluation, the linguistic evaluations by 
the professor suggesting to what extent each criterion meets in the 
project are presented in Table 2. 


Fuzzy number equivalents of linguistic variables 

The problem of project evaluation was attempted to be solved via 
“Extended Analysis Method” by Chang (1996). According to 
Chang’s method, the final conclusions are reached with the paired 
comparisons of both criteria and options with each other. Five 
linguistic variables are used, for comparing the students’ project 
evaluation criteria as “Absolutely Important,” “Very Strongly 
Important,” “Strongly/Essentially Important,” “Weakly Important,” 
and “Equally Important” according to a fuzzy five level scale (Chiou 
and Tzeng, 2002). The triangular fuzzy number for linguistic 
variables is defined by Erumit (2007). The triangular fuzzy number 
equivalents of linguistic variables used in evaluating main criteria, 
sub criteria and alternatives via paired comparison are presented in 
Table 3. 
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Table 1. Main/sub criteria and the weight of the criteria in the evaluation process. 


Criteria 

Code 

name 

Degree of 
importance 

Sub-criteria 

Code 

name 

Degree of 
importance 




Suitability for the purpose 

Ci 1 

V.l. 




Currency of knowledge 

C12 

Q.l. 

Content 

Cl 

V.l. 

Accuracy of knowledge 

C13 

1 . 




Suitability for spelling rules 

C14 

S.l. 




Suitability for user level 

C15 

1 . 




Simplicity 

C21 

S.l. 




Coherence 

C22 

1 . 

Design 

C2 

Q.I. 

Color harmony 

C23 

S.l. 




Layout and menu design 

C24 

1 . 




Harmony of multimedia elements 

C25 

V.l. 




Ease of use 

C31 

1 . 




Operating links properly 

C32 

1 . 

Technical 

C3 

1 . 

Operating pages properly 

C33 

S.l. 




Need for additional tech. 

C34 

S.l. 




Flexibility 

C35 

V.l. 




Having full knowledge of the topic 

C41 

V.l. 




Difficulty of application 

C42 

S.l. 

Presentation 

C4 

1 . 

Clarity and presentation capability 

C43 

1 . 




Presentation of the sources 

C44 

S.l. 




Use of time 

C45 

1 


Table 2. The project evaluation via linguistic variables. 

Criteria 

Project 1 

Project 2 

Project 3 

Project 4 

Project 5 

Ci i 

V.G 

V.G 

V.G 

V.G 

V.G 

Cl 2 

V.G 

G 

V.G 

V.G 

G 

Cl 3 

G 

G 

G 

V.G 

G 

Cl 4 

G 

G 

G 

V.G 

A 

C15 

G 

G 

V.G 

V.G 

A 

C21 

V.G 

G 

G 

V.G 

A 

C22 

G 

G 

V.G 

V.G 

G 

C23 

G 

V.G 

V.G 

V.G 

A 

C24 

G 

G 

V.G 

V.G 

G 

C25 

G 

G 

V.G 

V.G 

G 

C31 

V.G 

V.G 

V.G 

G 

G 

C32 

G 

V.G 

V.G 

G 

G 

C33 

G 

V.G 

V.G 

V.G 

V.G 

C34 

V.G 

G 

G 

V.G 

G 

C35 

V.G 

G 

G 

V.G 

P 

C4I 

V.G 

V.G 

V.G 

V.G 

A 

C42 

G 

A 

V.G 

V.G 

P 

C43 

G 

A 

G 

V.G 

G 

C44 

G 

A 

G 

G 

P 

C45 

G 

G 

A 

V.G 

G 
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Table 3. The triangular fuzzy number equivalents of linguistic variables used in the evaluation. 


Linguistic variable 

Triangular fuzzy number 

Reverse of triangular fuzzy number 

Equally important (E.l.) 

1,1,1 

1,1,1 

Weakly important (W.l.) 

0.5, 1.25, 2 

0.5, 0.8, 2 

Strongly important (S.l.) 

1.5, 2.25, 3 

0.33, 0.44, 0.66 

Very strongly important (V.l.) 

2.5, 3.25, 4 

0.25, 0.307, 0.40 

Absolutely important (A.I.) 

3.5, 4.25, 5 

0.20, 0.235, 0.285 


Table 4. Rule base of paired comparisons of fuzzy inputs related to criteria. 


Linguistic variable 

R.l 

S.l 

1 

Q.l 

V.l 

R.l 

E.l 

1/W.I 

1/S.I 

1/V.I 

1/A.I 

S.l 

W.l 

E.l 

1/W.I 

1/S.I 

1/V.I 

1 

S.l 

W.l 

E.l 

1/W.I 

1/S.I 

Q.l 

V.l 

S.l 

W.l 

E.l 

1/W.I 

V.l 

A.I 

V.l 

S.l 

W.l 

E.l 


Table 5. Rule base of paired comparisons of fuzzy inputs related to the project evaluation. 


Linguistic variable 

V.P 

P 

A 

G 

V.G 

V.P 

E.l 

1/W.I 

1/S.I 

1/V.I 

1/A.I 

P 

W.l 

E.l 

1/W.I 

1/S.I 

1/V.I 

A 

S.l 

W.l 

E.l 

1/W.I 

1/S.I 

G 

V.l 

S.l 

W.l 

E.l 

1/W.I 

V.G 

A.I 

V.l 

S.l 

W.l 

E.l 


Rule base of paired comparison 

Rule base is formed in order to determine the fuzzy exits which will 
be formed as a result of the comparison of criteria’s access input 
values. This rule base is prepared according to similar studies in the 
literature and the final version is formed by expert opinions. The 
rule base used in the paired comparison of the criteria is presented 
in Table 4. 

Since FAHP method bases on paired comparisons because of its 
structure, it requires to form a rule base in the comparison of the 
criteria. When the rule base in Table 5 is examined, it can be stated 
that the linguistic variables having similar degree of importance (If 
Ci=Very Important and C 2 =Very Important, the result of Paired 
Comparison is = Equally Important) are equally important and the 
degree of importance of linguistic variables having different degree 
of importance as a result of the paired comparison can be 
summarized as in Table 4. 

The rule base of paired comparison of linguistic variables such as 
very good, good, etc. used in the project evaluation is presented in 
Table 5. 

Similar to paired comparison of criteria, it is necessary to 
generate rule base for paired comparison of the students' projects. 
The rule base regarding linguistic variables employed for the 
evaluation of students' projects is presented in Table 5. 


RESULTS AND DISCUSSION 

Primarily, the weight of criteria which is gathered as a 


result of the paired comparison of main criteria and sub 
criteria was calculated, and then the paired comparison 
matrix which is formed as a result of the evaluation of 
each project according to these criteria, will be presented. 
Evaluators completed their assessments of relative 
importance for the criteria, paired comparison matrix for 
main criteria. The weight of criteria and the final grades 
obtained via the comparison of the projects are identified. 
The paired comparison matrix of priority values of main 
criteria according to the evaluator’s opinions is presented 
in Table 6. 

Taking into consideration the fuzzy number equivalents 
of the values expressed in Table 6 via linguistic variable 
regarding the result of the paired comparison of each 
main criterion, the data in Table 7 is obtained. 

Similarly, the paired comparison matrix of each sub 
criterion is given with the fuzzy number equivalents 
respectively. The paired comparison matrix of each sub 
criterion under the main content criteria is presented in 
Table 8. 

The degree of importance of the main design criteria of 
each sub criterion by the professor is determined as a 
result of paired comparison concerning primarily rule 
base, and then taking into consideration the fuzzy number 
equivalents of the identified the degree of importance, the 
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Table 6. The paired comparison matrix of main criteria 


Main Criteria 

Cl 

C 2 

C 3 

C 4 

Cl 

E.l 

W.l 

S.l 

S.l 

c 2 

1/W.I 

E.l 

W.l 

W.l 

c 3 

1/S.I 

1/W.I 

E.l 

E.l 

c 4 

1/S.I 

1/W.I 

E.l 

E.l 


Table 7. The paired comparison matrix of the fuzzy number equivalents of the main criteria. 


Main criteria 

Cl 

c 2 

c 3 

c 4 

Cl 

1,1,1 

0.5, 1.25, 2 

1.5, 2.25, 3 

1.5, 2.25, 3 

c 2 

0.5, 0.8, 2 

1,1,1 

0.5, 1.25, 2 

0.5, 1.25, 2 

C3 

0.33, 0.44, 0.66 

0.5, 0.8, 2 

1,1,1 

1,1,1 

c 4 

0.33, 0.44, 0.66 

0.5, 0.8, 2 

1,1,1 

1,1,1 


Table 8. The paired comparison matrix of main content criteria of each sub criterion. 


Main content criteria of each sub criterion 

Oil 

C 12 

C 13 

C 14 

C 15 

On 

1,1,1 

0.5, 1.25, 2 

1.5, 2.25, 3 

2.5,3.25,4 

1.5, 2.25, 3 

C 12 

0.5, 0.8, 2 

1,1,1 

0.5, 1.25, 2 

1.5, 2.25, 3 

0.5, 1.25, 2 

C 13 

0.33, 0.44, 0.66 

0.5, 0.8, 2 

1,1,1 

0.5, 1.25, 2 

1,1,1 

C 14 

0.25, 0.31,0.40 

0.33, 0.44, 0.66 

0.5, 0.8, 2 

1,1,1 

0.5, 0.8, 2 

C 15 

0.33, 0.44, 0.66 

0.5, 0.8, 2 

1,1,1 

0.5, 1.25, 2 

1,1,1 


paired comparison matrix of main design criteria of each 
sub criterion is found and presented in Table 9. 

The weight of criteria is determined as a result of the 
paired comparison of each criterion. Since a similar way 
is followed in the comparison of each criterion, only the 
comparison of the main criteria will be explained in detail 
here and when it comes to the comparisons of other 
criteria, only their results will be presented. The steps in 
the comparison of main criteria are as follows: 

Synthetic dimension values of paired comparisons of 
fuzzy evaluation matrix are calculated via equality (3) as 
follows: 

S(C0 = (4.50, 6.75, 9.00) ® (1/25.32; 1/17.53; 1/12.66) 
= (0.178, 0.385, 0.711) 

S(C 2 ) = (2.50, 4.30, 7.00) ® (1/25.32; 1/17.53; 1/12.66) 
= (0.099, 0.245, 0.553) 

S(C 3 ) = (2.83, 3.24, 4.66) 0 (1/25.32; 1/17.53; 1/12.66) 
= (0.112, 0.185, 0.368) 

S(C 4 ) = (2.83, 3.24, 4.66) ® (1/25.32; 1/17.53; 1/12.66) 

= (0.112, 0.185, 0.368) 


The probability of the expression, M 2 = (l 2 , m 2 , u 2 ) > Mi = 
(h, m,, u,) is defined by taking into account the Equation 
5: 

V(Sci > Sc 2 ) = 1 V(Sci > Sc 3 ) = 1 V(Sci > Sc 4 ) = 1 

V(S C2 >Sci)= 0.729 V(Sc 2 >S C3 ) = 1 V(S C2 >S C4 ) = 1 

V(Sc3>Sci)= 0.487 V(Sc3>S C2 ) = 0.817 V(Sc3>S C4 ) = 1 

V(S C4 > Sci) = 0.487 V(S C4 > S C2 ) = 0.817 V(S C4 > S C3 ) = 1 

With the help of these values, criteria are primarily 
calculated by using Equality 7 as follows: 

d^C,) = min (1,1,1) = 1 
d'(C 2 ) = min (0.729, 1,1) = 0.729 
d'(C 3 ) = min (0.487, 0.817, 1) = 0.487 
d'(C 4 ) = min (0.487, 0.817, 1) = 0.487 

The following vector is obtained as a result of the 
calculation of the priority vector: 
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Table 9. The paired comparison matrix of main design criteria of each sub criterion. 


Main design criteria of 
each sub criterion 

C21 

C22 

C23 

C24 

C25 

C21 

1,1,1 

0.5, 0.8, 2 

1,1,1 

0.5, 0.8, 2 

0.25, 0.307, 0.4 

C22 

0.5, 1.25, 2 

1,1,1 

0.5, 1.25, 2 

1,1,1 

0.33, 0.44, 0.66 

C23 

1,1,1 

0.5, 0.8, 2 

1,1,1 

0.5, 0.8, 2 

0.25, 0.307,0.4 

C24 

0.5, 1.25, 2 

1,1,1 

0.5, 1.25, 2 

1,1,1 

0.33, 0.44, 0.66 

C25 

2.5, 3.25, 4 

1.5, 2.25, 3 

2.5,3.25,4 

1.5, 2.25, 3 

1,1,1 


Table 10. The paired comparison matrix of main technical criteria of each sub criterion. 


Main technical criteria of 
each sub criterion 

C 31 

C 32 

C 33 

C 34 

C 35 

C 31 

1,1,1 

1,1,1 

0.5, 1.25, 2.0 

0.5, 1.25, 2.0 

1 , 1,1 

C 32 

1,1,1 

1,1,1 

0.5, 1.25, 2.0 

0.5, 1.25, 2.0 

1,1,1 

C 33 

0.5, 0.8, 2.0 

0.5, 0.8, 2.0 

1,1,1 

1,1,1 

0.5, 0.8, 2.0 

C 34 

0.5, 0.8, 2.0 

0.5, 0.8, 2.0 

1,1,1 

1,1,1 

0.5, 0.8, 2.0 

C 35 

1,1,1 

1,1,1 

0.5, 1.25, 2.0 

0.5, 1.25, 2.0 

1,1,1 


W'(1,0.729, 0.487, 0.487) 

Lastly, the obtained weight values are normalized and the 
final weights are determined: 

W=(0.370, 0.270, 0.180, 0.180) 

The weights obtained as a result of the paired comparison 
of main criteria prepared in order to evaluate the 
students’ projects, are as follows. The weight of content 
main criteria is 0.370; the weight of design main criteria, 
0.270; the weight of presentation criteria, 0.180; and the 
weight of technical main criteria, 0.180. Similarly, the 
weight vectors of paired comparison of all criteria are 
displayed in Table 12. 

After determining the weights related to the criteria, the 
evaluations of five projects under each criterion by 
decision maker are handled. The steps in determining the 
weights of the criteria are applied here as well. The 
weight vectors of the projects’ evaluation are presented in 
Table 13. 

The paired comparison matrix of main technical criteria 
of each sub criterion is displayed in Table 10. 

Similarly formed, the paired comparison matrix of main 
presentation criteria of each sub criterion is presented in 
Table 11. 

The results generated by multiplying the degree of 
importance of the main and sub criteria and the weight 
vectors of the students’ projects form the total weights of 
each criterion and these weights provide an indication of 
what extent a project meets the criteria compared to the 


other projects. It is possible to identify the total weight 
vectors of each project and accordingly the best project in 
Table 14. When the data in Table 14 are examined, it is 
determined that the best project is Project 4 and it is 
followed by Project 3, 1,2 and Project 5. 

We interviewed the professor about the evaluation 
result. He agreed that the evaluation result obtained by 
the developed fuzzy based evaluation system is more 
transparent and objective. The professor’s views on the 
results of the evaluation are as follows: 

1. Generally, I carry out the evaluation with numerical 
grades. Because converting linguistic variables into 
numerical grades is very difficult and takes a lot of time. 
However, with the help of this system, I was able to carry 
out linguistic evaluation and I got successful results. 

2. The developed system does not provide sharp lines 
but calculates intermediate values and provides a final 
result. It uses linguistic variables. I think, it is an 
applicable system and it will be a fair system. Also, while 
I am evaluating the students, I spend quite a lot of time to 
determine to what extend the students meet the criteria 
and also to calculate it. One of the benefits of this system 
is that it directly reveals the final results by carrying out 
the necessary calculations after your calculations... I 
think, this will contribute to the process of assessment 
and evaluation in education if it is used properly. 

3. Carrying out paired comparison and determining which 
group did a better job concerning each criterion are the 
advantages of this system compared to traditional 
evaluation. As human beings, we can compare the two 
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Table 11. The paired comparison matrix of main presentation criteria of each sub criterion. 


Main presentation criteria 
of each sub criterion 

C 41 

C 42 

C 43 

C 44 

C 45 

C41 

1,1,1 

2.5, 3.25, 4 

1.5, 2.25, 3 

2.5,3.25,4 

1.5, 2.25, 3 

C42 

0.25, 0.307, 0.4 

1,1,1 

0.5, 0.8, 2 

1,1,1 

0.5, 0.8, 2 

C43 

0.33, 0.44, 0.66 

0.5, 1.25, 2 

1,1,1 

0.5, 1.25, 2 

1,1,1 

C44 

0.25, 0.307, 0.4 

1,1,1 

0.5, 0.8, 2 

1,1,1 

0.5, 0.8, 2 

C45 

0.33, 0.44, 0.66 

0.5, 1.25, 2 

1,1,1 

0.5, 1.25, 2 

1,1,1 


Table 12. The weight vectors of main and sub criteria. 


Main criteria 

Weight vectors 

Sub-criteria 

Weight vectors 



On 

0.330 



C12 

0.243 

c, 

0.370 

C13 

0.153 



C14 

0.122 



C15 

0.153 



C21 

0.106 



C22 

0.133 

C2 

0.270 

C23 

0.106 



C24 

0.133 



C25 

0.521 



C31 

0.210 



C32 

0.210 

c 3 

0.180 

C33 

0.185 



C34 

0.185 



C35 

0.210 



C41 

0.521 



C42 

0.106 

c 4 

0.180 

043 

0.133 



C44 

0.106 



C45 

0.133 


groups with each other but the rate of making mistakes 
will increase when the number of groups increases. 
However, it is an advantage that this system makes this 
paired comparison automatically. In addition, I think that 
this system makes it possible to have more objective 
evaluations and to reach more accurate results. 


Conclusions 

One of the decision-making processes in education is the 
evaluation of the students’ projects. The evaluations in 
this process are generally carried out depending on the 
logic but intermediate evaluations are not taken into 
account. According to Qepni (2006), evaluating the 
students’ behaviors as black or white, or right or wrong in 


the evaluation process does not coincide with the modern 
education approach. 

In this study, since taking into consideration numerous 
criteria in the evaluation of students’ projects and the 
question of to what extend these criteria are met involve 
ambiguity, it is stated that it requires an intense mental 
effort. The formation of the proposed system on the basis 
of fuzzy set theory determines that it can provide benefits 
in modeling these ambiguities in human mental 
processes and also it can be reached fairer, more 
sensitive and more objective results. In this respect, this 
study shares similarities with the studies of Lin (2010), 
Montero et al. (2005), and Saleh and Kim (2009). 

The studies in literature indicate that it is difficult to 
evaluate the projects with numerical data and using 
linguistic variables is more beneficial (Chang and Sun, 
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Table 13. The weight vectors of the projects. 


Criteria 

Project 1 

Project 2 

Project 3 

Project 4 

Project 5 

On 

0.200 

0.200 

0.200 

0.200 

0.200 

Cl2 

0.210 

0.185 

0.210 

0.210 

0.185 

Cl 3 

0.191 

0.191 

0.191 

0.237 

0.191 

Cl4 

0.193 

0.193 

0.193 

0.246 

0.176 

Cl5 

0.191 

0.191 

0.234 

0.234 

0.152 

C21 

0.233 

0.191 

0.191 

0.233 

0.152 

C22 

0.188 

0.188 

0.219 

0.219 

0.188 

C23 

0.191 

0.233 

0.233 

0.233 

0.110 

C24 

0.188 

0.188 

0.219 

0.219 

0.188 

C25 

0.188 

0.188 

0.219 

0.219 

0.188 

C3I 

0.210 

0.210 

0.210 

0.185 

0.185 

C32 

0.188 

0.219 

0.219 

0.188 

0.188 

C33 

0.179 

0.205 

0.205 

0.205 

0.205 

C34 

0.219 

0.188 

0.188 

0.219 

0.188 

C35 

0.273 

0.221 

0.221 

0.273 

0,010 

C41 

0.250 

0.250 

0.250 

0.250 

0.000 

C42 

0.225 

0.134 

0.290 

0.290 

0.060 

C43 

0.193 

0.176 

0.193 

0.246 

0.193 

C44 

0.233 

0.191 

0.233 

0.233 

0.110 

C45 

0.193 

0.193 

0.176 

0.246 

0.193 


Table 14. The final results of the projects’ evaluations. 

Main criteria weight 
vector 

Sub-criteria weight 
vector 

Project 1 

Project 2 

Project 3 

Project 4 

Project 5 


On 

0.330 

0.200 

0.200 

0.200 

0.200 

0.200 


Cl 2 

0.243 

0.210 

0.185 

0.210 

0.210 

0.185 

p 

o 

GO 

O 

C13 

0.153 

0.191 

0.191 

0.191 

0.237 

0.191 


Cl4 

0.122 

0.193 

0.193 

0.193 

0.246 

0.176 


C15 

0.153 

0.191 

0.191 

0.234 

0.234 

0.152 


C 2 I 

0.106 

0.233 

0.191 

0.191 

0.233 

0.152 


C22 

0.133 

0.188 

0.188 

0.219 

0.219 

0.188 

C 2 0.270 

C23 

0.106 

0.191 

0.233 

0.233 

0.233 

0.110 

C24 

0.133 

0.188 

0.188 

0.219 

0.219 

0.188 


C25 

0.521 

0.188 

0.188 

0.219 

0.219 

0.188 


C31 

0.210 

0.210 

0.210 

0.210 

0.185 

0.185 


C32 

0.210 

0.188 

0.219 

0.219 

0.188 

0.188 

C 3 0.180 

C33 

0.185 

0.179 

0.205 

0.205 

0.205 

0.205 


C34 

0.185 

0.219 

0.188 

0.188 

0.219 

0.188 


C35 

0.210 

0.273 

0.221 

0.221 

0.273 

0.010 


C41 

0.521 

0.250 

0.250 

0.250 

0.250 

0.000 


C42 

0.106 

0.225 

0.134 

0.290 

0.290 

0.060 

C 4 0.180 

C43 

0.133 

0.193 

0.176 

0.193 

0.246 

0.193 


C44 

0.106 

0.233 

0.191 

0.233 

0.233 

0.110 


C45 

0.133 

0.193 

0.193 

0.176 

0.246 

0.193 

Total weight vectors 



0.206 

0.200 

0.215 

0.225 

0.156 

Rank 



3 

4 

2 

1 

5 
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1993; Law, 1996). FAHP based system allows decision 
makers to evaluate via linguistic variables. That this 
system processes the linguistic variables via converting 
them into fuzzy numbers and provides a final weight 
score is interpreted as a positive feature by the professor. 
Also, it is found that the results of the system are 
satisfactory. The followings are stressed as the 
disadvantages of traditional evaluation methods: It is 
difficult to carry out paired comparison among projects 
when the number of projects increases; it takes a lot of 
time to carry out numerical procedures for each project; 
and there can occur probable errors in the evaluation 
process. Thanks to the developed FAHP based system, it 
is possible to follow the same steps for all projects, to 
provide errorless calculations, and to make complicated 
calculation quickly. In addition, with the FAHP method, 
the paired comparisons of both criteria and projects can 
be achieved automatically. As a result of this feature, 
more accurate results in the evaluation of projects can be 
obtained. Also this approach can reduce subjectivity in 
the evaluation process. In this study, another point to be 
emphasized is that the application of fuzzy logic 
principles in the process of the project evaluation 
provides the desired flexibility for the existing system. 
The sensitivity of the evaluation of the target population 
can be increased by defining the criteria and the degree 
of importance in detail, thanks to the flexible structure. 
This study supports researchers and practitioners an 
insight into how FAHP can be used in evaluating student 
projects. 

This study has some limitations that can be considered 
as recommendations for future studies. First, this study 
focus on that FAHP method can be used in the 
evaluation of students’ projects in education. In future 
studies, other multi-criteria decision making methods 
such as fuzzy TOPSIS, fuzzy ANP, fuzzy PROMETHEE, 
fuzzy ELECTRE and their modification can be used for 
similar applications and the results obtained are 
comparable. Second, the evaluation criteria were 
determined as the result of interviews with experts. 
Future research can use different methodologies, such as 
longitudinal studies or a review of the literature to identify 
evaluation criteria. Finally, this study was conducted with 
relatively small samples. Similar studies can be 
conducted on larger sample and generalizability can be 
increased. 
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